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Abstract 

A path information is defined in connection with the probabihty 
distribution of paths of nonequihbrium hamiltonian systems moving 
in phase space from an initial cell to different final cells. On the basis 
of the assumption that these paths are physically characterized by 
their action, we show that the maximum path information leads to 
an exponential probability distribution of action which implies that 
the most probable paths are just the paths of stationary action. We 
also show that the averaged (over initial conditions) path information 
between an initial cell and all the possible final cells can be related 
to the entropy change defined with natural invariant measures for 
dynamical systems. Hence the principle of maximum path information 
suggests maximum entropy and entropy change which, in other words, 
is just an application of the action principle of classical mechanics to 
the cases of stochastic or instable dynamics. 

1 Introduction 

The principle of maximum information and entropy are investigated for 
nonequilibrium hamiltonian system in connection with action principle of 
classical mechanics. 

The entropy defined by Clausius is an equilibrium quantity[l] which are 
related later by Boltzmann and Gibbs to equilibrium probability distributions 
with S" = Inf and S = — Z^lLi P« ^^Pi' respectively, here Pi is the normalized 
(J21=iPi = 1) probability for the system to be found at state i, v is the total 
number of states in the phase space occupied by the system (let Boltzmann 
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constant be unity). The methodology of the statistical mechanics based on 
Boltzmann and Gibbs entropies is later complemented by the principle of 
maximum entropy [1, 2] which stipulates that the "best" equilibrium prob- 
ability distributions should maximize Clausius entropy. This method was 
already used by Boltzmann and Gibbs[l, 3]. But thanks to the efforts of 
Jaynes[l 1], it has been nowadays widely recognized to be a powerful infer- 
ence method for guessing (not deriving) correct probability distributions of 
equilibrium as well as nonequilibrium systems. It should be noticed that one 
of the underlying philosophical significations of this method is the anthropo- 
morphic aspect of entropy and information. They are not objective physical 
quantities. They are related to the ignorance of the observers. They can be 
used only for guessing, but not deriving probability[l 1]. 

On the other hand, compared to equilibrium systems, the statistical and 
informational methodology for dynamical (nonequilibrium) systems seems 
less certain. We have seen recently the extension of the Boltzmann and Gibbs 
entropies to nonequilibrium systems (in local equilibrium or not) [4, 5, 6]. But 
as for the inference or guessing methods of nonequilibrium distributions, al- 
though it was assumed by Jaynes that the maximum entropy method always 
applied, the principle of minimum entropy production [7] has been proposed 
for the systems sufficiently close to equilibrium state. For certain systems 
far from equilibrium, e.g., in the context of the climate of earth, we have 
still the principle of maximum entropy production (or minimum entropy 
exchange) [8, 9]. Here the entropy is always in the sense of Clausius, so the 
entropy production dSi > according to the second law. Over the last years, 
we have seen the development of a nonextensive statistics which suggests 
that the Havrda-Charvat-Daroczy-Tsallis entropy[10] should be maximized 
for nonequilibrium systems[ll, 12, 13], a new extension of Jaynes method to 
dynamical systems. 

So we have before us many different even opposite methods for dynamical 
systems. Why do we have to maximize or minimize entropy or its produc- 
tion for equilibrium as well as nonequilibrium systems in variational inference 
method? Are information and entropy merely anthropomorphic quantities 
which have nothing to do with physical laws? Indeed, Jaynes method is not 
theoretically proved in spite of its success in "guessing" probability distribu- 
tions. It is only supported by plausible arguments such as, e.g., "if a system 
is in a state with lower entropy it would contain more information than pre- 
viously specified", or "higher entropy states are more probable" and "less 
predictable", etc[l, 14]. So maximum entropy is only considered as a princi- 
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pie of guess, but not deduction method of mechanics. The central quantities 
of this method, i.e., entropy and information, are only anthropomorphic [14], 
not objective quantities involved in fundamental laws of physics. 

In this work, Jaynes approach is revisited under a different angle in con- 
nection with the principle of least (stationary) action. It will be shown that, 
for stochastic or instable dynamics of hamiltonian systems, maximum infor- 
mation and entropy must be used to derive correct probability distributions 
just as the least (stationary) action principle must be used for regular dy- 
namics to derive the correct motions. We finally reach the conclusion that 
the anthropomorphic aspect of entropy and information is not needed in or- 
der to be able to maximize them for inference and correct guess. Maximum 
entropy is in fact a deductive method for deriving probability distributions. 
It is an application of the fundamental action principle of classical mechanics 
to probabilistic dynamics. 

2 Assumptions and definitions 

Let us begin by some assumptions and definitions concerning dynamical sys- 
tems. 

1. We recall that a phase space F of thermodynamic system is defined such 
that a physical state of the system is represented by a point in that 
space. A phase volume Q can be partitioned into v cells of volume Si 
with i = 1,2, ...V in such a way that Si fl Sj = (i 7^ j) and U^^iSi = il. 
A state of the system can be represented by a sufficiently small phase 
cell in coarse graining way. The movement of a dynamical system can 
be represented by its trajectories (in the sense of classical mechanics) 
in r space. 

2. The natural invariant measure[15] fii is defined as the probability dis- 
tribution for a nonequilibrium system to visit different elementary cells 
i of its partitioned phase space. 

3. The information we address in this work is a measure of the uncertainty 
of dynamical process. According to Shannon, this information can be 
calculated by the formula 

V 

H = -"^Pklnpk (1) 

k=l 
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with respect to certain probability distribution of that process and 
the index k is summed over all the possible situations. 

4. The entropy S* of a nonequilibrium system is defined with the natural 
invariant measure /ij by 

V 

= -^/iiln/ii. (2) 

If this system reaches equilibrium at the end of a nonequilibrium pro- 
cess and has its final probability distribution, then S can be 
considered as the entropy in the sense of second law (Clausius). 



3 A path information 

In a previous work [16], we defined a path information for dynamical systems 
moving, over a large period of time, in the F-space between two points, a and 
6, which are in two elementary cells of a given partition of the phase space. 
This path information is given by the Shannon formula with respect to the 
probability pfc(6|a) for the system to take the path k from the cell a to the 
cell b during tab{k), as shown in Figure 1(1). 

By definition, pk{b\a) is a transition probability from state a to state b 
via path k. We have Z)fc=iPfc(^|a) = 1- This path probability distribution 
due to dynamical instability is studied in connection with information theory 
and action integral on the basis of the assumption that the different paths 
are uniquely differentiated by their actions defined for classical mechanical 
systems by 

Aabik) = [ Lk{t)dt (3) 

Jtab{k) 

where Lk[t) is the Lagrangian of the system at time t along the path k. The 
average action between state a and state b can be given by 

w 

Aab = Y.Pk{b\a)Aab{k) (4) 
fc=i 

For all stochastic process like Brownian motion[r7], it can be proved[16] 
that the most probable paths are just the paths of least action. The proba- 
bility of other paths decreases exponentially with increasing action. 
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In this work, we introduce another approach in order to study a more 
general situation where the systems under consideration moves from a initial 
cell a to different destinations h with a given travelling time, as shown in 
Figure 1(11). This "spacial uncertainty" and the "time uncertainty" studied 
in [16] are two aspects of the same dynamical instability. That is, for regular 
motion, these two uncertainties both disappear. 

Now let us consider an ensemble of large N identical systems leaving the 
initial cell a in the initial phase volume A for some destinations in the phase 
volume B formed by the final phase points occupied by the systems. The 
travelling time is tab = tb—ta fixed for every trajectory and destination. After 
tab, all the phase points occupied by the systems are found in the volume B 
partitioned into cells labelled by b. We observe A^^^^^ systems travelling along 
a path kb leading to certain cell b. A path probability can be defined by 
Pki,\a = Nk^/N which is normalized by 

overB uij 

12 Y Pkt\a = 1- (5) 
b ki=l 

where Wb is the number of possible paths from a to a given cell b of the 
volume B. We always suppose each path is characterized by its action Ai^^\a- 
Then an average action can be defined by 

overB 

^a= YPkb\aAk,\a (6) 

b kh=l 

The uncertainty concerning the choice of paths and final cells is measured by 
the following Shannon information 

overB 

HB\a = - Y YPhla'^'^Pkbla (7) 

b kf, 

which can be maximized under the constraints associated with Eq.(5) and 
Eq.(6) as follows 

overB Wf, overB lOj, 

6{HB\a + a Y Y Ph\a + V Y Y Pkt\aAk,\a) = 0. (8) 
b fci,=l b fci,=l 

This leads to 

Pkb\a = ^ exp[-r]Ak,\a], (9) 
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where Z = J^k, exp[-r/Afc^|J. Putting Eq.(9) back into Eq.(7), we get 

HB\a = ^nZ + 7]Aa = \nZ-7]-^\nZ. (10) 

It is proved that [16] the distribution Eq.(9) is stable with respect to the 
fluctuation of action. It is also proved that if rj is positive, Eq.(9) is a least 
action distribution, i.e., the most probable paths are just the paths of least 
action. On the contrary, if rj is negative, then Eq.(9) is a most action distri- 
bution which means that the most probable paths maximize action. In any 
case, whatever the sign of the parameter rj, the most probable paths which 
maximize the path information always correspond to extremum or station- 
ary action {SAf^^^a = 0)- I''^ other words, for instable dynamical process, the 
method of maximum information must be used in order to derive correct prob- 
ability distribution just as the principle of stationary action must be used to 
derive the correct trajectories for regular dynamics. 

In our previous work[16], Brownian motion was presented as an example 
of the case of 77 > 0. We would like to mention in passing that this model 
of Brownian motion allowed us to give a simple derivation of Pick law for 
particle diffusion and of Pourrier law for heat conduction in solids[lS] thanks 
to the distribution function Eq.(9). 



4 Maximum entropy change 

The question here is how to derive the invariant measures /ij with which the 
entropy 5* for dynamical systems is defined through Eq.(2). 

In what follows, we suppose that entropy changes when a system moves 
from the initial cell a to the final cell b. The initial entropy of the systems 
at time t = is given by Shannon formula: 

Sa = -^fia^l^fla (H) 

a 

where fia is the invariant measure of the systems in the initial phase volume 
A at t = and the cell index a is summed over all the occupied cells in A. 
After a period of time tab, the system travelling along a path kb is found in a 
cell b with probability measure fik,,{o-)- Supposing the transition probability 
is independent of the distribution fia of initial conditions, we have 

= /^aPfctk- (12) 
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The entropy of the system after tab should be defined by 



over A overB 



(13) 



b fcfc 



over A 



overA overB 
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a b 
Sa + Hb\A- 



a 



where Hb\a is the average of the path information Hi,\a over all the initial 
phase volume A. So if HB\a is maximized, Hb\a should also be maximized. 
The sums in the above equation must be over all the cells in the initial phase 
volume A, in the final phase volume B (formed by all cells b) and over all 
the possible paths between A and B. 
Eq.(13) can be written as 



where (In Z) and (A) are the averages of In Z and Aa on the volume A. 
Eq.(14) is the main result of this work. It is obvious that the maximization 
of Hb\a means the maximization of the entropy change AS" of the dynamical 
process. 

5 What about maximum entropy? 

Maximum entropy change for dynamical systems does not exclude the possi- 
bility of maximum entropy. In fact, if the entropy change is maximized at any 
moment of an evolution with respect to different probability distributions, 
given the initial entropy Sa, the entropy Si, of the final states must also be 
at maximum all the time with respect to the same distributions. So in order 
to derive correct probabilities or invariant measures, both entropy change 
and entropy can be maximized, depending on which of the two quantities is 
available as function of the probability distributions to be derived. 

If the initial phase volume A and the final volume B represent equilibrium 
states, i.e., the dynamical process connects two Clausius entropies Sa and Sb, 
then Boltzmann and Gibbs formula can be maximized to derive equilibrium 
probability distributions pi of final states, as suggested by Jaynes principle. 



AS = Sb-Sa = Hb\a = (InZ) + r,{A) 



(14) 
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This can be considered as a derivation of the maximum entropy method from 
the maximum path information which is nothing but an apphcation of the 
action principle of classical mechanics. 

6 Concluding remarks 

A path information is defined in connection with different possible paths of 
dynamical system moving in its phase space from the initial cell towards dif- 
ferent final cells. On the basis of the assumption that the paths are physically 
differentiated by their actions, we show that the maximum path information 
leads to a path probability distribution implying that the most probable 
paths are just the paths of stationary action. It is also shown that the aver- 
age path information defined in this work is just the variation of the entropy 
during the dynamical process. Hence the principles of least action and of 
maximum path information suggest the maximum entropy change for dy- 
namical systems. For given initial distribution, maximum entropy change 
means in fact the maximization of final entropy with respect to the same 
probability distributions. 

We would like to mention that the intrinsic link between entropy and 
action tells us that the information theory and statistical mechanics do not 
need that entropy be an anthropomorphic quantity. Information and entropy 
are nothing but objective measures of dynamical uncertainty when the mo- 
tion of physical systems become probabilistic. So maximum information and 
entropy is not merely an inference principle. It is a law of physics. This is 
the main conclusion of this paper. 
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Figure 1: (I) The possible phase space paths (A; = 1, 2...w) for a system to go 
from the cell a to the cell h in time tab{k), each having a probability Pab{k). 
If w is the number of all the possible paths, we have Y^^=i'Pab{k) = 1. (II) 
The possible phase space paths for a system to go from a given cell a to the 
different cells 6 of a partition of the final phase volume B during the time 
tab, each having a probability Pk^ia- More than one paths from a to a given 
cell b are allowed so we may have kb = 1, 2...Wb where Wb is the total number 
of paths from a to a cell b. 
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